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Preface  &  Acknowledgements 


Welcome  to  our  Ninth  Annual  Acquisition  Research  Symposium!  This  event  is  the 
highlight  of  the  year  for  the  Acquisition  Research  Program  (ARP)  here  at  the  Naval 
Postgraduate  School  (NPS)  because  it  showcases  the  findings  of  recently  completed 
research  projects — and  that  research  activity  has  been  prolific!  Since  the  ARP’s  founding  in 
2003,  over  800  original  research  reports  have  been  added  to  the  acquisition  body  of 
knowledge.  We  continue  to  add  to  that  library,  located  online  at 

www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  60  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  hope  this  symposium  will  spark  even  more  participation. 

We  encourage  you  to  be  active  participants  at  the  symposium.  Indeed,  active 
participation  has  been  the  hallmark  of  previous  symposia.  We  purposely  limit  attendance  to 
350  people  to  encourage  just  that.  In  addition,  this  forum  is  unique  in  its  effort  to  bring 
scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  Seldom  will  you  get  the  opportunity  to  interact  with  so 
many  top  DoD  acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both 
in  the  formal  panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals, 
breaks,  and  the  day-ending  socials.  Many  of  our  researchers  use  these  occasions  to 
establish  new  teaming  arrangements  for  future  research  work.  In  the  words  of  one  senior 
government  official,  “I  would  not  miss  this  symposium  for  the  world  as  it  is  the  best  forum 
I’ve  found  for  catching  up  on  acquisition  issues  and  learning  from  the  great  presenters.” 

We  expect  affordability  to  be  a  major  focus  at  this  year’s  event.  It  is  a  central  tenet  of 
the  DoD’s  Better  Buying  Power  initiatives,  and  budget  projections  indicate  it  will  continue  to 
be  important  as  the  nation  works  its  way  out  of  the  recession.  This  suggests  that  research 
with  a  focus  on  affordability  will  be  of  great  interest  to  the  DoD  leadership  in  the  year  to 
come.  Whether  you’re  a  practitioner  or  scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 

•  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology,  &  Logistics) 

•  Director,  Acquisition  Career  Management,  ASN  (RD&A) 

•  Program  Executive  Officer,  SHIPS 

•  Commander,  Naval  Sea  Systems  Command 

•  Program  Executive  Officer,  Integrated  Warfare  Systems 

•  Army  Contracting  Command,  U.S.  Army  Materiel  Command 

•  Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
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•  Office  of  the  Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  & 
Technology) 

•  Deputy  Director,  Acquisition  Career  Management,  U.S.  Army 

•  Office  of  Procurement  and  Assistance  Management  Headquarters,  Department 
of  Energy 

•  Director,  Defense  Security  Cooperation  Agency 

•  Deputy  Assistant  Secretary  of  the  Navy,  Research,  Development,  Test  & 
Evaluation 

•  Program  Executive  Officer,  Tactical  Aircraft 

•  Director,  Office  of  Small  Business  Programs,  Department  of  the  Navy 

•  Director,  Office  of  Acquisition  Resources  and  Analysis  (ARA) 

•  Deputy  Assistant  Secretary  of  the  Navy,  Acquisition  &  Procurement 

•  Director  of  Open  Architecture,  DASN  (RDT&E) 

•  Program  Executive  Officer,  Littoral  Combat  Ships 

We  also  thank  the  Naval  Postgraduate  School  Foundation  and  acknowledge  its 
generous  contributions  in  support  of  this  symposium. 

James  B.  Greene  Jr.  Keith  F.  Snider,  PhD 

Rear  Admiral,  U.S.  Navy  (Ret.)  Associate  Professor 
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Sean  F.  Crean — Mr.  Crean  is  the  director  of  the  Office  of  Small  Business  Programs  for  the 
Department  of  the  Navy.  He  serves  as  chief  advisor  to  the  Secretary  on  all  small  business  matters. 

He  is  responsible  for  small  business  acquisition  policy  and  strategic  initiatives. 

Mr.  Crean  joined  the  Secretary  of  the  Navy  Staff  as  a  member  of  the  Senior  Executive  Service  in 
January  2010  and  has  over  30  years  of  federal  service.  Prior  to  receiving  this  appointment,  he  served 
as  Deputy  Assistant  Secretary  of  the  Navy  for  Acquisition  and  Logistics  Management  during  a  two- 
year  military  recall  to  active  duty  as  a  rear  admiral  in  support  of  Operation  Iraqi  Freedom. 

Mr.  Crean’s  previous  experience  includes  serving  as  the  senior  procurement  analyst  for  the  U.S. 
Small  Business  Administration’s  Office  of  Government  Contracting  Area  I  (New  England)  for  19 
years.  In  this  role  he  was  the  principal  advisor  to  the  SBA’s  six  regional  district  offices  and 
congressional  delegations  on  procurement  issues.  He  provided  acquisition  strategy  analysis  for  over 
20  buying  activities  throughout  the  region,  supporting  both  DoD  and  civilian  federal  agencies.  He  first 
entered  federal  civilian  service  as  the  deputy  supply  officer  for  Naval  Air  Station  Brunswick,  ME, 
where  he  was  also  appointed  the  activity  small  business  specialist. 

Mr.  Crean’s  combined  military  and  civil  service  careers  have  provided  complimentary  and 
extensive  leadership  responsibilities  in  service  to  the  country.  As  a  member  of  the  reserve 
component,  he  has  attained  the  grade  of  rear  admiral  (two-star)  and  is  currently  assigned  as  deputy 
commander,  Naval  Supply  Systems  Command.  He  holds  a  Bachelor  of  Science  degree  in  business 
management  and  marine  transportation  from  State  University  of  New  York  Maritime  College  and  a 
Master  of  Business  Administration  degree  from  New  Hampshire  College’s  Graduate  School  of 
Business. 

He  has  a  number  of  personal  and  command  decorations,  including  two  Legion  of  Merit  awards. 
He  is  a  member  of  the  Defense  Acquisition  Corps  and  is  DAWIA  Level  III  Contracting  certified. 
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Improving  the  Methodology  for  Setting  Small  Business 

Size  Thresholds1 
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with  the  state  of  California,  [nancy@rand.org] 
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labor  market  inequalities,  and  public  assistance  programs.  Her  recent  research  examines  the  role  that 
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small  businesses  in  the  marketplace  and  by  women  and  racial-ethnic  minorities  in  the  workplace. 
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Lloyd  Dixon — Dixon  is  a  senior  economist  at  RAND  specializing  in  microeconomics  and  empirical 
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projects  and  has  studied  regional  and  local  economic  issues  and  policies.  Dr.  Dixon  holds  a  BS  in 
engineering  and  a  BA  in  political  science  from  Stanford  University,  and  a  PhD  in  economics  from  the 
University  of  California  at  Berkeley,  [dixon@rand.org] 
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holds  bachelor's,  master's,  and  doctoral  degrees  from  the  University  of  Chicago  in  political  science. 
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Judith  Mele — Mele  is  a  research  programmer  at  RAND  where  she  analyzes  large-scale,  complex 
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Abstract 

The  Small  Business  Administration  (SBA)  recently  developed  a  new  method  for  determining 
whether  a  business  is  small  or  other-than-small  for  procurement  purposes.  The  resulting  firm- 
size  thresholds  determine  whether  a  business  is  eligible  for  federal  procurement  preferences, 
as  well  as  whether  the  Department  of  Defense  meets  its  statutory  goals  for  direct  contract 
dollars  with  small  businesses.  The  definition  of  what  goods  and  services  represent  an 
industry,  as  well  as  what  metric  the  SBA  should  use  to  measure  firm  size,  affects  the 
outcome  of  the  method,  as  does  the  data  that  are  used  for  it.  If  the  industry  definition  is  too 
broad  or  narrow,  if  the  metric  is  inappropriate  for  the  industry,  or  if  the  data  is  flawed  because 
of  how  it  is  collected,  the  size  threshold  will  be  inappropriate.  A  method  that  more  directly 
assesses  industry  characteristics,  as  well  as  reassesses  the  industry  definition  and  metric 
used  to  measure  firm  size,  would  help  improve  the  quality  of  the  size-thresholds 
determination  process. 


1  This  paper  summarizes  work  that  was  done  for  the  DoD  Office  of  Small  Business  Programs  within  RAND’s 
National  Defense  Research  Institute  Federally  Funded  Research  and  Development  Center. 
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Introduction 

The  federal  government  has  long  sought  to  boost  small  businesses  by  purchasing 
goods  and  services  from  them.  Currently,  there  is  a  government-wide  goal  that  the  federal 
government  spend  23%  of  its  dollars  for  goods  and  services  directly  with  small  businesses. 
This  goal  reflects  federal  policy  that  a  “fair  proportion”  of  all  federal  purchases  should  be 
made  with  small  businesses. 

Such  a  goal  raises  the  question  of  what  a  “small”  business  is.  For  procurement 
preferences  and  other  programs  such  as  loan  assistance,  the  Small  Business  Administration 
(SBA)  sets  size  thresholds  by  industry.  Today,  these  thresholds  are  typically  500  employees 
for  manufacturing  industries  and  $7  million  in  annual  receipts  for  services  firms. 

The  Department  of  Defense  (DoD)  accounts  for  about  two  thirds  of  all  federal 
purchases  and  hence  is  key  to  meeting  the  government-wide  goal  for  purchases  from  small 
businesses.  The  pattern  of  DoD  purchases  by  industry  can  make  this  difficult. 

For  example,  the  DoD  spends  more  money  on  aircraft  manufacturing  than  in  any 
other  industry,  as  Table  1  shows.  Only  about  2%  of  its  dollars  in  this  industry  are  with  small 
businesses.  This  roughly  approximates  the  share  that  small  firms  have  of  the  industry  but 
makes  it  difficult  for  the  DoD  to  have  23%  of  all  its  purchases  be  with  small  businesses. 

Table  1.  DoD  Success  in  Meeting  23%  Goal  Varies  by  Industry 

DoD  Success  in  Meeting  23  Percent  Goal  Varies  by 
Industry-  Selected  Examples 


Industry 

Small  business  percent 

2002  Economic 
Census 

DoD  2002 
utilization 

Aircraft  Manufacturing  (336411) 

2.6% 

1.8% 

Guided  Missile  and  Space  Vehicle 
Manufacturing  (336414) 

2.3 

1.9 

Military  Armored  Vehicle,  Tank,  and  Tank 
Component  Manufacturing  (336992) 

15.1 

3.1 

Computer  Systems  Design  Services 
(541512) 

28.1 

26.8 

Other  Computer  Related  Services 
(541519) 

45.1 

25.4 

Facilities  Support  Services  (561210) 

15.6 

34.2 

RAND  it  ju».ia 

3curaw  AJLU  Ccsncmc  Can  mum.  hiwari  ywr r  AMIi  JU23J  JMbi 


In  five  of  the  top  ten  industries  in  which  the  DoD  spends  money,  small  businesses 
account  for  less  than  16%  of  all  industry  activity.  Complicating  what  may  be  a  small 
business  within  a  given  industry  is  the  extraordinary  variety  of  goods  and  services  that  may 
be  sold  within  it.  For  example,  aircraft  manufacturing  includes  manufacturing  of  aircraft, 
autogiros,  blimps,  gliders,  helicopters,  and  ultra-light  aircraft,  goods  of  varying  interest  to  the 
DoD  and  amenability  to  small-business  production. 
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These  issues  led  the  DoD  Office  of  Small  Business  Programs  to  ask  the  RAND 
Corporation  to  review  issues  regarding  size  standards.  This  paper  briefly  summarizes  the 
establishment  and  evolution  of  size  standards,  how  large  and  small  businesses  view 
threshold  issues,  and  what  improvements  might  be  made  to  the  process  for  setting 
standards. 

Establishment  and  Evolution  of  Standards:  Arbitrary  Origins  and  Failure  to 
Keep  Pace  With  Industry  Changes? 

Efforts  to  define  small  businesses  date  at  least  to  the  1930s,  when  the 
Reconstruction  Finance  Corporation  needed  to  set  a  threshold  for  determining  eligibility  for 
loans  to  small  businesses.  Efforts  to  determine  them  for  military  procurement  purposes  date 
at  least  to  World  War  II,  when  the  Smaller  War  Plants  Corporation  set  a  threshold  of  500 
employees  for  determining  what  constitutes  a  small  business.  This  threshold  was 
considered  “arbitrary”  by  two  historians  but  persisted  through  the  enactment  of  the  Small 
Business  Act  in  1953. 

The  Small  Business  Act  in  1953  gave  to  the  SBA  discretion  to  set  size  thresholds. 
Thresholds  for  most  industries  are  set  to  what  the  SBA  calls  “anchor”  standards:  500 
employees  for  manufacturing  firms  and  currently  $7  million  in  average  annual  receipts  for 
services  firms.  Other  legislation  has  used  other,  typically  far  smaller,  thresholds  to  define 
small  businesses  for  non-procurement  purposes,  as  Table  2  illustrates. 

Table  2.  Congress  Has  Used  Many  Different  Size  Thresholds  for  Applicability  of 

Legislation 

Congress  Has  Used  Many  Different  Size  Thresholds 
for  Applicability  of  Legislation 


Act 

Year 

Threshold 

Civil  Rights 

1964 

15  employees 

Age  Discrimination  in  Employment 

1967 

20  employees 

Occupational  Safety  and  Health 

1970 

11  employees 

COBRA  (on  group  health  plans) 

1986 

20  employees 

Worker  Adjustment  and  Retraining 

1988 

100  employees 

Americans  with  Disabilities 

1990 

15  employees 

Clean  Air  Amendments 

1990 

100  employees 

Family  and  Medical  Leave 

1993 

50  employees 

FDA  Modernization 

1997 

$500K  in  sales 

RAND 


The  SBA  has  adjusted  thresholds  over  time,  but  often  not  as  fast  as  economic 
conditions  have  changed.  Although  more  manufacturing  is  now  concentrated  among  firms  of 
at  least  500  employees,  the  SBA  has  not  changed  the  anchor  threshold  of  500  employees.  It 
has  adjusted  the  anchor  receipts  threshold  for  services  firms  over  time,  but  not  at  the  pace 
of  inflation  indices,  as  shown  in  Figure  1 . 
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Figure  1.  SBA  Anchor  Revenue  Thresholds  Have  Not  Always  Kept  Pace  With 

Economic  Conditions 


Business  Views  of  Size-Threshold  Issues:  Confusion  Over  Industry  Definitions 
and  Thresholds 

Representatives  of  large  and  small  businesses  that  we  interviewed  identified  several 
problems  with  current  size  thresholds.  Large-business  representatives  noted  that  high 
capital  needs,  costly  production  inputs,  salaries  for  professional  employees,  and  security 
requirements  can  all  boost  revenue  requirements  and  push  small  businesses  over  the  size 
threshold.  Penalties  for  inadvertently  and  incorrectly  identifying  a  business  as  small,  these 
representatives  told  us,  can  sometimes  lead  them  to  avoid  small  subcontractors  for  certain 
federal  government  work.  Emerging  best  practices  for  fewer,  larger  suppliers,  with  smaller 
suppliers  pushed  further  up  the  supply  chain,  have  also  made  it  more  difficult  for  large  prime 
contractors  to  gain  subcontracting  credit  for  federal  contract  work. 

The  small-business  representatives  we  interviewed  also  noted  problems  with  costs 
that  could  boost  total  revenues  but  not  profits.  They  contended  that  dichotomous  size 
categories  (i.e. ,  small  and  other-than-small)  for  federal  small-business  programs  do  not 
adequately  reflect  the  incremental  growth  that  their  businesses  undergo.  They  reported 
confusion  over  inconsistent  industry  definitions  and  thresholds  and  contended  fluctuations  in 
federal  business  can  make  it  difficult  to  forecast  demand  and  growth. 

Improving  Methods  for  Determining  What  Constitutes  a  Small  Business: 

Justify  Anchor  Standards,  Define  Industries,  and  Measure  Better 

Further  complicating  the  establishment  of  small-business  size  thresholds  are  issues 
with  the  data  used  to  set  them.  Table  3  contains  different  sources  of  data  on  the  number  of 
firms,  establishments,  employees,  and  receipts,  showing  differing  sizes  of  industries. 
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Table  3.  Differing  Sources  Can  Have  Widely  Varying  Data 

Differing  Sources  Can  Have  Widely  Varying  Data  - 
Example:  Ammunition  Manufacturing  (332993) 


Published 
Economic 
Census  Data 

Special 
Tabulations 
for  SBA 

Statistics  of 
U.S. 

Businesses 

FPDS 

Firms 

n/a 

49 

46 

102 

Establishments 

54 

54 

52 

139 

Employees 

7,111 

6,925 

5,984 

n/a 

Receipts  ($s  M) 

1,128 

1,190 

875 

1,887 

RAND 


Further,  federal  procurement  data  in  some  industries,  such  as  manufacturing  of 
ammunition  and  armored  vehicles,  reports  that  the  value  of  DoD  purchases  exceeds  the 
size  of  the  industry  as  the  Census  Bureau  reported,  as  shown  in  Table  4.  Better-quality  data 
could  help  determine  the  size  and  shape  of  each  industry  and  what  constitutes  small 
business  within  it. 
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Table  4.  Data  Used  for  Setting  Thresholds  May  Also  Fail  to  Reflect  True  Conditions 

in  Defense  Industries 

Data  Used  for  Setting  Thresholds  May  Also  Fail  to 
Reflect  True  Conditions  in  Defense  Industries 


Calendar/Fiscal  Year 
2002 

Ammunition 
(except  small  arms) 
Manufacturing1 

Military  Armored  Vehicle, 
Tank,  and  Tank  Component 
Manufacturing2 

Reported  industry 
dollars  ($sB)3 

$1.19 

$1.51 

DoD  goaling  dollars 
($s  B)* 

$1.89 

$1.94 

Reported  %  small- 
business  in  industry3 

28.1% 

10.5% 

DoD  %  dollars  to 
small-business  in 
industry* 

12.2% 

3.1% 

'  NAICI  uks: 

;  NAICI  «4«2 
'-20G2  Ecorwric  Census 

RAND  •fPO* 

Seuraw  AUJ  teoflomc  Can  mux.  '  xcM  y«r  AVI  UJ33U 


Firm  reporting  of  revenue  by  “primary”  NAICS  may  also  lead  to  distortions  in  the 
distribution  of  firms  by  size  in  an  industry.  The  notional  example  in  Table  5  illustrates  how 
firm  X  with  multiple  establishments  producing  in  one  or  more  industries  can  report  very 
different  revenues  per  industry,  depending  on  the  level  at  which  it  reports  and  the 
distribution  of  revenue  among  NAICS. 
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Table  5.  Fidelity  of  Firm  Reporting  Can  Lead  to  Under  and  Over  Estimates  of 

Revenue  for  Each  NAICS 
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Fidelity  of  Firm  Reporting  Can  Lead  to  Underand 
Over  Estimates  of  Revenue  for  Each  NAICS 


Firm  X 

Business  units 

NAICS  revenue  reporting  by 

NAICS 

Establishment 

Firm 

Establishment  A 

NAICS  1 

500,000 

1,000,000 

NAICS  2 

300,000 

NAICS  3 

200,000 

Total  Revenue 

1,000,000 

Establishment  B 

NAICS  4 

1,200,000 

2,000,000 

NAICS  5 

800,000 

Total  Revenue 

2,000,000 

Establishment  C 

NAICS  6 

5,000,000 

5,000,000 

8,000,000 

Total  Firm  Revenue 

8,000,000 

RAND 


Better  justification  of  the  anchor  standards  may  also  help.  These  are  based  on 
economic  conditions  and  the  state  of  technology  at  least  60  years  ago  and  were  established 
in  part  for  want  of  a  better  definition.  No  analytic  case  has  been  made  for  them;  such  might 
provide  insight  into  what  should  constitute  a  small  business  for  procurement  and  other 
purposes. 

Industries  can  also  be  very  broad,  even  at  the  level  of  six-digit  North  American 
Industry  Classification  System  code  as  illustrated  in  Figure  2.  A  given  industry  may 
encompass  more  activities  than  are  of  interest  to  the  DoD.  Conversely,  DoD  purchases  in 
some  industries  can  encompass  a  wide  range  of  activities.  More  narrow  industry  definitions 
may  be  needed  to  capture  more  homogeneous  activities. 
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An  Industry  Can  Be  Defined  Very  Broadly 

•  A  single  industry  can  include  very  different  types  of 
goods  or  services 

-  Aircraft  Manufacturing  (336411)  includes  aircraft, 
blimp,  helicopter,  ultra-light,  and  hang  glider 
manufacturing 

-  Engineering  Services  (541330)  includes  acoustical, 
boat,  chemical,  civil,  construction,  heating,  mining, 
and  traffic  engineering  services 

•  To  increase  competitiveness  in  some  industries 

-  One  size  standard  may  not  be  appropriate 
exceptions 

-  Industry  may  need  to  be  more  narrowly  defined 

RAND 


Figure  2.  An  Industry  Can  Be  Defined  Very  Broadly 

Measures  for  some  industries  may  also  be  inappropriate.  For  example,  receipts  may 
not  be  an  appropriate  measure  for  service  industries  with  high  capital  requirements,  high 
material  costs,  highly  paid  employees,  or  expensive  certification  requirements.  The  SBA 
also  has  an  upper  bound  on  size  standards;  higher  size  standards  than  this  level  may  be 
needed  to  achieve  the  competitiveness  goals  of  small-business  policy. 

The  SBA  recently  revised  its  methodology  for  setting  small  business  size  thresholds, 
which  is  summarized  in  Figure  3.  We  identified  a  number  of  shortcomings  with  the 
methodology.  First,  there  is  no  justification  for  the  anchor  standards  with  which  it  starts  to 
compare  industries.  The  groups  of  industries  it  compares  to  one  another  may  be 
inappropriate  because  of  basic  structural  differences.  As  mentioned  earlier,  industries  can 
be  very  broad.  The  size  metric  the  SBA  uses  may  no  longer  be  appropriate  for  the  industry. 
The  SBA  sets  an  arbitrary  upper  bound  on  its  size  standards.  Lastly,  the  comparison  metrics 
that  it  does  use  are  not  all  supported  by  analysis. 
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Overview  of  SBA’s  Size  Standard  Methodology 


RAND 


Figure  3.  Overview  of  SBA’s  Size  Standard  Methodology 

An  ideal  approach  to  estimating  size  thresholds  would  directly  asses  the  following 
five  industry  characteristics. 

First,  the  method  should  identify  the  minimum  firm  size  needed  to  remain  competitive 
in  an  industry.  Average  costs  of  production  typically  fall  as  production  increases.  These  can 
vary  by  industry.  Small-business  size  thresholds  might  be  based  in  part  on  the  size  required 
to  be  an  ongoing  concern  in  a  given  industry. 

Second,  a  method  for  determining  what  a  small  business  is  should  identify  the  size 
below  which  access  to  credit  becomes  more  limited.  Access  to  credit  is  critical  to  operations 
of  a  firm.  Future  thresholds  might  consider  credit  terms  available  to  firms  of  different  sizes  in 
an  industry. 

Third,  the  method  should  identify  the  minimum  size  needed  to  administer  a  federal 
prime  contract  for  the  types  of  goods  and  services  purchased  by  the  military  from  the 
industry.  To  successfully  compete  for  federal  contracts,  a  business  must  be  of  sufficient  size 
to  administer  them.  Accordingly,  a  method  for  setting  small-business  size  by  industry  should 
consider  the  resources  and  number  of  personnel  needed  to  administer  the  types  of 
contracts  typically  issued  by  the  federal  government  in  a  particular  industry. 

Fourth,  it  should  evaluate  criteria  based  on  the  competitiveness  of  the  industry.  The 
Herfindahl-Hirshman  Index  (HHI)  may  be  more  appropriate  for  this  than  the  Gini  coefficient 
the  SBA  currently  uses.  The  Federal  Trade  Commission  uses  the  HHI,  a  ratio  showing 
concentration  of  industry  revenues,  in  determining  whether  to  approve  mergers  or 
acquisitions. 

Finally,  different  weights  of  these  criteria  may  be  appropriate  for  differing  purposes. 
The  size  standards  derived  from  each  of  the  above  criteria  will  likely  differ,  leaving 
policymakers  to  determine  which  is  most  relevant  in  a  given  industry.  To  satisfy  all, 
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policymakers  can  pick  the  maximum  size  threshold  suggested  by  individual  criteria  in  setting 
a  threshold  for  defining  small  business. 

We  propose  an  alternative  new  methodology  for  setting  size  threshold,  which  is 
summarized  in  Figure  4.  It  addresses  some  of  the  data  shortcomings  discussed  earlier,  such 
as  an  inhomogeneous  NAICS  and  inappropriate  size  measure,  as  well  as  shortcomings  in 
the  current  SBA  methodology  and  attempts  to  directly  assess  the  above  five  industry 
characteristics. 
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Figure  4.  Overview  of  Proposed  New  Size  Threshold  Methodology 

In  summary,  we  found  that  the  economic  census  and  FPDS  data  supporting  SBA 
threshold  analyses  are  flawed.  Current  threshold  metrics  used  for  specific  industries  may  no 
longer  be  appropriate,  and  some  industry  classifications  may  be  too  broad.  The  current  SBA 
methodology  for  setting  thresholds  has  a  number  of  weaknesses.  Lastly,  the  SBA  has  no 
methodology  for  targeting  industries  for  threshold  review. 

We  recommend  that  the  SBA  work  to  improve  the  quality  of  economic  census  and 
FPDS  data,  and  that  before  changing  thresholds,  the  SBA  should  analyze  an  industry’s 
metric  to  see  if  it  is  still  appropriate,  as  well  as  the  industry’s  classification  to  see  if  it  is  too 
broad  or  too  narrow.  The  SBA’s  methodology  should  acknowledge  the  unique  and  changing 
aspects  of  industries  that  are  likely  to  decouple  it  from  its  past  thresholds,  as  well  as  those 
of  other  industries.  Lastly,  the  SBA  should  develop  and  refine  a  methodology  for  targeting 
industries  for  metric,  classification,  and  threshold  review,  such  as  focusing  on  industries  with 
threshold  exceptions,  which  suggest  that  the  metric  or  classification  may  need  changing, 
and  survey  large  and  small  businesses  to  identify  any  industries  where  they  are 
encountering  problems  with  current  thresholds. 
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Federal  Government  Seeks  to  Boost  Procurement 

from  "Small"  Businesses 

*  Small  Business  Act  seeks  “to  insure  that  a  fair 
proportion  of  the  total  purchases  ...  for  the 
Government ...  be  placed  with  small  businesses]” 

*  Government-wide  goal  since  1997  is  for  23  percent  of 
“prime”  contract  dollars  to  be  with  small  businesses 

*  Department  of  Defense  (DoD)  accounts  for  about  two- 
thirds  of  all  federal  purchases  and  is  key  to  meeting 
government-wide  goal 

*  Threshold  for  small-business  size  varies  by  industry 
defined  by  North  American  Industry  Classification 
System  (NAICS) 
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Congress  Has  Used  Many  Different  Size  Thresholds 

for  Applicability  of  Legislation 


Act 

Year 

Threshold 

Civil  Rights 

1964 

15  employees 

Age  Discrimination  in  Employment 

1967 

20  employees 

Occupational  Safety  and  Health 

1970 

11  employees 

COBRA  (on  group  health  plans) 

1986 

20  employees 

Worker  Adjustment  and  Retraining 

1988 

100  employees 

Americans  with  Disabilities 

1990 

15  employees 

Clean  Air  Amendments 

1990 

100  employees 

Family  and  Medical  Leave 

1993 

50  employees 

FDA  Modernization 

1997 

$500K  in  sales 

RAND 
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The  DoD  Office  of  Small  Business  Programs  Asked 
RAND  to  Review  Issues  Regarding  Size  Thresholds 

*  Establishment  and  evolution  of  size  thresholds 

*  How  large  and  small  businesses  view  size 
threshold  issues 

*  Data  issues 

*  Possible  improvements  to  SBA  method  for  setting 
size  thresholds 
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Small  Business  Administration  (SB A)  Has 
Responsibility  to  Set  Thresholds  for  Its  Programs 

*  Small  Business  Act  of  1953  gave  SBA  administrator 
discretion  to  set  size  thresholds 

*  Initial  criteria  were  that  a  firm  be 

-  Independently  owned  and  operated 

-  Not  dominant  in  its  field  of  operations 

*  For  procurement,  these  guidelines  led  to  anchor 
standards  of 

-  500  employees  in  most  manufacturing  industries 

-  $1  million  (now  $7  million)  in  most  non¬ 
manufacturing  industries 
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Small  Business*  Share  of  Economy  Has  Decreased 
Since  Thresholds  First  Set,  Particularly  for  Services 
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*Small  business  defined  as  firm  with  fewer  than  500  employees 
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SBA  Anchor  Revenue  Thresholds  Have  Not  Always 
Kept  Pace  with  Economic  Conditions 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


RAND 


•Federal  statute 
Inflated  1963  $s 
Inflated  1976  $s 
Inflated  1985  $s 
Inflated  1995  $s 
Inflated  2003  $s 
Inflated  2006  $s 
Inflated  2009  $s 


/ 


J  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  L 


J  I  I  I  I  I  I  I  I  I  I 


MlflNOrrtlflNOrCOinNOrCOlONO) 

0(O0<ONNSNNOOCOOOOOOOO)a)OO)O 

000000000000000000)0 


Fiscal  year 


n  m  s  a) 


CM  CM  CM  CM  CM  CM 


Thresholds  are  as  reported  in  13  CFR  §121,  Small  Business  Size  Regulations,  annual.  Inflated  dollars  calculated  using  2012 
deflators  in  U.S.  Department  of  Defense,  National  Defense  Budget  Estimates  for  FY  2013,  March  2012. 
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Marine  Engineering  Thresholds  Have  Also  Changed \ 
but  Not  Always  at  Pace  of  Economic  Conditions 
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Thresholds  are  as  reported  in  13  CFR  §121,  Small  Business  Size  Regulations,  annual.  Inflated  dollars  calculated  using  2012 
deflators  in  U.S.  Department  of  Defense,  National  Defense  Budget  Estimates  for  FY  2013,  March  2012. 
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The  DoD  Office  of  Small  Business  Programs  Asked 
RAND  to  Review  Issues  Regarding  Size  Thresholds 

*  Establishment  and  evolution  of  size  thresholds 

*  How  large  and  small  businesses  view  size 
threshold  issues 

*  Data  issues 

*  Possible  improvements  to  SBA  method  for  setting 
size  thresholds 
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We  Interviewed  Large  and  Small  Businesses 
Regarding  Their  Experiences  with  Size  Thresholds 

*  Interviews 

-  Lasted  about  45  minutes  each 

-  Followed  set  protocol  of  open-ended  questions 

*  Large  businesses 

-  Firms  with  extensive  federal  contracting  experience, 
most  in  the  DoD 

-  Small  business  representative 

-  Nine  companies  invited  to  participate;  4  agreed 

*  Small  businesses 

-  In  engineering  industry 

-  16  companies  invited  to  participate;  2  agreed 
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Large  Business  Representatives  in  Our  Interviews 
Noted  Nine  Obstacles  to  Finding  Small  Partners 

*  Factors  that  push  businesses  over  revenue  thresholds 

-  Capital  needs 

-  Costly  inputs 

-  Certification  requirements 

-  Employee  skills 

*  Operational  challenges  to  using  small  businesses 

-  Lead  Systems  Integrator  (LSI)  trend 

-  Emerging  best  practices  in  purchasing  and  supply 
management 

-  Growing  quickly  beyond  threshold 

-  Acquisition  of  innovative  small  businesses 

-  Methods  for  applying  industry  size  exceptions 
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Obstacles  to  Finding  Small  Partners  Are  Particularly 

Great  in  Some  Industries 

*  Information  and  technology 

*  Architectural  and  engineering  services 

*  Legal  services 

*  Accounting 

*  Aircraft  manufacturing 


Most  companies  reported  difficulty  in  finding 
small  businesses  in  some  industries 
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Small  Business  Representatives  in  Our  Interviews 
Noted  Five  Challenges  to  Being  DoD  Contractors 

1 .  Requirements  that  boost  revenue  but  not  profit 

2.  Only  two  size  categories:  small  and  other-than- 
small 

3.  Inconsistent  industry  definitions  and  thresholds 

4.  Costs  of  government-required  certification 

5.  Large  shifts  in  federal  spending 
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Implications  from  Interviews  for 
Small-Business  Policy 

*  Large  and  small  businesses  identified  multiple 
problems  with  revenue  based  thresholds 

*  Both  suggested  thresholds  based  on  number  of 
employees  are  often  better  metrics  of  size 

*  Incremental  growth  of  business  not  reflected  in 
policy  ^  more  threshold  levels  could  help  address 
this 
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The  DoD  Office  of  Small  Business  Programs  Asked 
RAND  to  Review  Issues  Regarding  Size  Thresholds 

*  Establishment  and  evolution  of  size  thresholds 

*  How  large  and  small  businesses  view  size 
threshold  issues 

*  Data  issues 

*  Possible  improvements  to  SBA  method  for  setting 
size  thresholds 


RAND 


15  Jun-10 


DRAFT— DO  NOT,  COPY,  QUOTE,  OR  CITE 

DoD  Success  in  Meeting  23  Percent  Goal  Varies  by 

Industry  -  Selected  Examples 


Industry 

Small  business  percent 

2002  Economic 
Census 

DoD  2002 
utilization 

Aircraft  Manufacturing  (336411) 

2.6% 

1 .8% 

Guided  Missile  and  Space  Vehicle 
Manufacturing  (336414) 

2.3 

1.9 

Military  Armored  Vehicle,  Tank,  and  Tank 
Component  Manufacturing  (336992) 

15.1 

3.1 

Computer  Systems  Design  Services 
(541512) 

28.1 

26.8 

Other  Computer  Related  Services 
(541519) 

45.1 

25.4 

Facilities  Support  Services  (561210) 

15.6 

34.2 

RAND 


Sources:  2002  Economic  Census,  Fiscal  year  2002  DD350  data 
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Data  Used  for  Setting  Thresholds  May  Also  Fail  to 
Reflect  True  Conditions  in  Defense  Industries 


Calendar/Fiscal  Year 
2002 

Ammunition 
(except  small  arms) 
Manufacturing1 

Military  Armored  Vehicle, 
Tank,  and  Tank  Component 
Manufacturing2 

Reported  industry 
dollars  ($sB)3 

$1.19 

$1.51 

DoD  goaling  dollars 
($8  B)* 

$1.89 

$1.94 

Reported  %  small- 
business  in  industry3 

28.1% 

10.5% 

DoD  %  dollars  to 
small-business  in 
industry* 

12.2% 

3.1% 

1  NAICS  332993 

2  NAICS  336992 

32002  Economic  Census 

*2002  DD350  data 
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Available  Data  Sources  Differ  in  How  They  Gather 
and  Report  Identical  Firms  in  Differing  Industries 

•  Gather  by 

-  Establishment  of  firm 

*  Statistics  on  U.S.  Businesses  (SUSB) 

*  Economic  Census  (EC) 

*  Special  Size  Tabulations  for  SBA  (SBA  tabs) 

*  Federal  Procurement  Data  System  (FPDS)  by  industry 
reported  on  contract  action 

-  Firm  -  Central  Contractor  Registry  (CCR)  -  industries  in 
which  firm  seeks  to  provide  goods  and  services 

•  Reports  primarily  by 

-  Establishment  -  Published  EC 

-  Firm 

.  SUSB 

*  SBA  tabs 

*  FPDS 

Different  gathering  and  reporting  ^  miscounting 
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Differing  Sources  Can  Have  Widely  Varying  Data  - 
Example:  Ammunition  Manufacturing  (332993) 


Published 
Economic 
Census  Data 

Special 
Tabulations 
for  SB  A 

Statistics  of 
U.S. 

Businesses 

FPDS 

Firms 

n/a 

49 

46 

102 

Establishments 

54 

54 

52 

139 

Employees 

7,111 

6,925 

5,984 

n/a 

Receipts  ($s  M) 

1,128 

1,190 

875 

1,887 
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Fidelity  of  Firm  Reporting  Can  Lead  to  Under  and 
Over  Estimates  of  Revenue  for  Each  NAICS 


Firm  X 

Business  units 

NAICS  revenue  reporting  by 

NAICS 

Establishment 

Firm 

Establishment  A 

NAICS  1 

500,000 

1,000,000 

NAICS  2 

300,000 

NAICS  3 

200,000 

Total  Revenue 

1,000,000 

Establishment  B 

NAICS  4 

1,200,000 

2,000,000 

NAICS  5 

800,000 

Total  Revenue 

2,000,000 

Establishment  C 

NAICS  6 

5,000,000 

5,000,000 

8,000,000 

Total  Firm  Revenue 

8,000,000 
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An  Industry  Can  Be  Defined  Very  Broadly 

•  A  single  industry  can  include  very  different  types  of 
goods  or  services 

-  Aircraft  Manufacturing  (336411)  includes  aircraft, 
blimp,  helicopter,  ultra-light,  and  hang  glider 
manufacturing 

-  Engineering  Services  (541330)  includes  acoustical, 
boat,  chemical,  civil,  construction,  heating,  mining, 
and  traffic  engineering  services 

*  To  increase  competitiveness  in  some  industries 

-  One  size  standard  may  not  be  appropriate  ^ 
exceptions 

-  Industry  may  need  to  be  more  narrowly  defined 
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Poor  Correlation  Between  NAICS  and  Product  Service 
Codes  (PSCs)  in  FPDS  Indicates  Further  Imprecision 

Engineering  Services 

-  2007  NAICS  Engineering  Services  (  541330)  has 
640  PSCs  associated  with  it 

-  PSC  Engineering  and  Technical  Services  (R425) 
has  144  2007  NAICS  associated  with  it 

Facilities  Support  Services 

-  2007  NAICS  Facilities  Support  Services  (561210) 
has  328  PSCs  associated  with  it 

-  PSC  Facilities  Operations  Support  Services 
(S216)  has  161  NAICS  associated  with  it 


RAND 


Source:  FY2010  FPDS 
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The  DoD  Office  of  Small  Business  Programs  Asked 
RAND  to  Review  Issues  Regarding  Size  Thresholds 

*  Establishment  and  evolution  of  size  thresholds 

*  How  large  and  small  businesses  view  size 
threshold  issues 

*  Data  issues 

*  Possible  improvements  to  SBA  method  for  setting 
size  thresholds 

-  Current  SBA  methodology 

-  Shortcomings  of  SBA  approach 

-  Proposed  improved  methodology 
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Overview  of  SB  A ’s  Size  Standard  Methodology 
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SBA  Weighting  Method  Starts  with  Anchor  Groups 

*  500  employees  for  manufacturing  industries 

*  $7  million  in  annual  revenues  for  service  industries 

*  100  employees  for  wholesale  trade  industries 
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SB  A  Periodically  May  Update  Threshold  After 
Comparison  to  Other  Industries 

Compares 

1 .  Industry  anchor  size  standard 

2.  Industries  with  higher-size  standards  that  were 
created  by  past  adjustments  of  industry  size 
standards 

-  For  receipts-based  standards,  industries  with 
size  standards  ranging  from  $23  million  to 
$35.5  million 

-  For  employee-based  standards,  industries  with 
a  1,000-employee  standard 
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Shortcomings  of  SBA  Approach 
for  Revising  Standards 

*  No  justification  for  anchor  standards 

*  Comparison  groups  may  be  inappropriate 

*  Industry  can  be  very  broad 

*  Size  metric  may  be  inappropriate 

*  Upper  bound  on  size  standards 

*  Possibly  irrelevant  comparison  metrics  not 
supported  by  analysis 
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An  Ideal  Approach  to  Setting  Industry  Size  Standards 
Would  Directly  Assess  Industry  Characteristics 

*  Minimum  efficient  production  level 

*  Size  below  which  access  to  credit  becomes 
significantly  more  limited 

*  Minimum  size  needed  to  successfully  execute  a 
federal  prime  contract  for  the  types  of  goods  and 
services  purchased 

*  Herfindahl-Hirschman  Index  (HHI)  of  concentration 
throughout  industry 
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Overview  of  Proposed  New  Size  Threshold  Methodology 


Proposed  new  threshold 
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Targeting  Industries  for  Threshold  Revision: 

Four  Possible  Methods 

1.  Industries  with  significant  changes  in  firm  distribution 
between  EC 

2.  Industries  with  firms  reported  in  FPDS  moving  above 
thresholds  or  back  and  forth  across  thresholds 

3.  Industries  where  percent  federal  FPDS  goaling  dollars  to 
small  firms  is  significantly  less  than  reported  percent  small 
business  revenue  in  the  EC 

4.  Industries  where 

•  Large  prime  contractors  report  having  difficulty  finding 
qualified  small  businesses 

•  Small  prime  contractors  and  subcontractors  report 
having  problems  staying  below  thresholds 
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Findings 


*  Data  supporting  SBA  threshold  analyses  is  flawed 

-  Economic  Census 

-  FPDS 

*  Current  threshold  metrics  used  for  specific  industries  may  no 
longer  be  appropriate 

*  Some  industry  classifications  may  be  too  broad 

*  SBA  methodology  for  setting  thresholds  has  a  number  of 
weaknesses 

*  SBA  has  no  methodology  for  targeting  industries  for 
threshold  review 
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Recommenda  tions 


*  Work  to  improve  the  quality  of  economic  census  and  FPDS  data 

*  Before  changing  threshold,  the  SBA  should  analyze  industry 

-  Metric  for  its  appropriateness 

-  Classification  regarding  whether  it  is  too  broad  or  too  narrow 

*  Improve  methodology  to  acknowledge  unique  and  changing  aspects 
of  industries 

*  Develop  and  refine  a  methodology  for  targeting  industries  for  metric, 
classification,  and  threshold  analyses 

-  Industries  with  exceptions  suggests  that  the  metric  or 
classification  may  need  changing 

-  Survey  large  and  small  businesses  to  identify  industries  and 
their  problems 
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Questions? 
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